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ABSTRACT 

While institutional researchers commonly evaluate the 
programs and services provided by their colleges, they rarely evaluate their 
own effectiveness with as much gusto. If Institutional Research (IR) is to 
serve as the bedrock of an institution 1 s ef f ectiveness , it must be effective 
in itself and continually improving. It must make certain that the services 
it provides meet the needs of its customers. To this end a comprehensive 
Institutional Research Effectiveness Plan (IR-EP), rooted in the mission and 
goals of the college, was created, used, and evaluated. The results were 
almost as interesting as the process itself. The 5-Column Model of J. Nichols 
was used to create the IR-EP, and objectives were written that encapsulated 
what IR at this college was attempting to do to meet each goal. IR closely 
examined its role at the college with a focus on how people should benefit 
from what was done. An interview with the dean and survey of customers 
(college faculty and personnel) showed that IR data were used extensively at 
the college, but that while customers clearly understood IR reports, they 
wanted an increased emphasis on application and recommendation. The IR-EP, 
designed to promote continuous improvement, can be used to help the college 
carry out its mission and achieve its goals. (Author/SLD) 
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THE IR-EP: A POWERFUL MODEL FOR SELF-ASSESSMENT AND PLANNING 

IN IR 

Abstract 

While institutional researchers commonly evaluate the programs and services provided by 
their colleges, they rarely evaluate their own effectiveness with as much gusto. If Institutional 
Research is to serve as the bedrock of an institution's effectiveness, it must be effective and 
continually improving. It must make certain that the services it provides meet the needs of its 
customers. To this end, a comprehensive Institutional Research Effectiveness Plan, rooted in the 
mission and goals of the college, was created and utilized. The results were almost as interesting 
as the process itself. 



Overview 

The 5-Column Model, developed by James O. Nichols, is a template for measuring and 
promoting effectiveness. The model can be utilized at the institutional level, or applied to 
instructional or administrative units. It consists of five major components: 1) Mission and Goals, 
2) Intended Outcomes/Objectives, 3) Means of Assessment and Criteria for Success, 4) Results, 
and 5) Use of Results. 



Mission and Goals 

The 5-Column Model was used to create an Institutional Research Effectiveness Plan (IR- 
EP) based on the mission and goals of the college. IR goals were derived from the college's 
goals. As such, IR had to determine how it related to and supported those goals. For instance, 
one of the college's goals was to "provide academic programs for an associate degree in the arts 
and sciences." IR determined that its corresponding goal was to "support the college in its 
endeavors to provide academic programs for an associate degree in the arts and sciences." While 
some college units support the mission and goals directly, through teaching and student services, 
IR offices are "twice removed." Nonetheless, while IR seldom serves students directly, it serves 
those who serve students. 



Objectives 

Objectives were written that encapsulated what IR was attempting to do to meet each goal. 
The objectives were written using outcomes-oriented language, focusing on how the receiver of 
IR information and services should benefit. The driving question was, how is each customer able 
to do her or his job better because of the information provided by IR? This customer-centered 
approach forced heightened accountability. IR felt increased pressure to become more 
responsible for the data it produced and the manner in which it shared those data with others. 

IR closely examined its role at the college during this phase. The focus switched from "this 
is what I do" to "this is how people should benefit from what I do." For instance, an objective 
was crafted which stated, "The college will be well equipped to provide appropriate degree and 
transfer programs because of the information provided by IR." It was no longer enough to 
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conduct research and produce reports. The true measure of success became the degree to which 
individuals used the information produced by IR to further the mission and goals of the college. 

Means of Assessment and Criteria for Success 

Once objectives were established, they were operationalized. Objectives outlined a) where 
the data were to come from (the assessment method that would be used) and b) what would 
constitute acceptable performance. 

IR typically utilized several assessment methods (both qualitative and quantitative) to 
measure its effectiveness in each area. For instance, IR determined that an interview with the 
Dean of Instruction would indicate the degree to which the college was equipped to provide 
appropriate degree and transfer programs because of the data provided by IR. IR also stated that 
it would receive a mean score of at least 3.0 on "the college is well equipped to offer appropriate 
academic programs because of the information provided by IR" on an IR customer survey. 

Summary of Data Collected 

Numbers were recorded for quantitative measures, while summaries were written for 
qualitative methods. A written report summarized the interview held with the Dean of 
Instruction. In short, the dean felt that IR data were used extensively to make improvements in 
existing academic programs, but that more visible data were needed to guide the creation of new 
programs at the college. 

A customer survey was utilized to procure much of the quantitative data needed for the IR- 
EP. It was found that IR received an average rating of 3.3 (out of 4) in this area. The results of 
both assessment measures converged to tell the same story: IR was doing a good job in this area, 
but there were opportunities for improvement. 

Data collection was enjoyable and educational. The value of sitting down and talking with 
key decision makers and leaders about the usefulness of IR data cannot be overstated. Two major 
revelations occurred. First, IR learned that its data were used extensively at the college. It 
learned, for example, that it was common practice in certain division meetings to study IR 
reports and use the data to improve performance in their area. Second, IR discovered that while 
its customers clearly understood its reports, they desired an increased emphasis on application 
and recommendation. As one interviewee stated, "We want you to hit us over the head with it." 

Use of Results 

While the data collection phase required summarization and synthesis, the final phase 
comprised analysis and planning. In other words, IR asked itself, what does this mean and what 
am I going to do about it? It was no coincidence that the IR-EP was completed in time for IR to 
plan its activities for the following year. 

IR studied the amassed data collectively and identified strategies that would address as many 
areas as possible at the same time. IR discovered, for instance, that data in several parts of the 
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IR-EP pointed to the need for information regarding the needs of the community. As a result, a 
Community Needs Survey was planned. Similarly, the need for more blatantly applicable 
information was uncovered in several areas. As such, plans were made to hit them over the head 
with it during the following year. 



Conclusion 

The IR-EP is a cyclical progression. The amassing of data marks the beginning and not the 
end of the cycle. The Use of Results phase (column 5) is a planning phase. By this point, IR has 
done certain activities (conducted research, produced reports, etc.), assessed its effectiveness, 
adjusted its thinking and its priorities, and is now ready to plan its future activities in light of 
what it has learned. 

Last but certainly not least, the ultimate purpose of the IR-EP was not performance appraisal 
or evaluation. While self-evaluation certainly occurred, it occurred en route to planning. The 
results of assessment were used by IR to make improvements in its services. The goal of the IR- 

EP then, is the continuous improvement of Institutional Research, which subsequently 
assists the college in more successfully carrying out its mission and achieving its goals. 
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